On the Identifiability of Overcomplete Dictionaries via the Minimisation Principle Underlying K-SVD

This article gives theoretical insights into the performance of K-SVD, a dictionary learning algorithm that has gained significant popularity in practical applications. The particular question studied here is when a dictionary Φ ∈ Rd×K can be recovered as local minimum of the minimisation criterion underlying K-SVD from a set of N training signals yn = Φxn. A theoretical analysis of the problem leads to two types of identifiability results assuming the training signals are generated from a tight frame with coefficients drawn from a random symmetric distribution. First asymptotic results showing, that in expectation the generating dictionary can be recovered exactly as a local minimum of the K-SVD criterion if the coefficient distribution exhibits sufficient decay. This decay can be characterised by the coherence of the dictionary and the `1-norm of the coefficients. Based on the asymptotic results it is further demonstrated that given a finite number of training samples N , such that N/ logN = O(Kd), except with probability O(N−Kd) there is a local minimum of the K-SVD criterion within distance O(KN−1/4) to the generating dictionary.